Skip to content

chore(spec): clean fellowship sediment + retry commands eventually#212

Merged
nijeesh-stream merged 2 commits into
masterfrom
nijeeshjoshy/chore-fix-queries-users-flake
May 13, 2026
Merged

chore(spec): clean fellowship sediment + retry commands eventually#212
nijeesh-stream merged 2 commits into
masterfrom
nijeeshjoshy/chore-fix-queries-users-flake

Conversation

@nijeesh-stream
Copy link
Copy Markdown
Contributor

@nijeesh-stream nijeesh-stream commented May 12, 2026

Summary

Three pre-existing flakes in client_spec.rb all caused by the shared CI test app — none are SDK behavior bugs.

1. queries users — sediment + concurrent runs

before(:all) seeds a fellowship of four named users (Frodo Baggins, Samwise Gamgee, Gandalf the Grey, Legolas) and hard-deletes them in after(:all). When a CI run dies between those hooks — which has happened repeatedly across recent webhook + JWT PRs — the leftovers stay on the shared app. The next run's queries users test pins race = 'Hobbit' length to 2 and picks up sediment: got: 6 (or 8).

Fix (two-pronged):

  • Active cleanup at the top of before(:all)query_users by the four fellowship names and hard-delete any leftovers via delete_users(ids, user: HARD_DELETE, messages: HARD_DELETE). Idempotent; safe whether or not the previous run cleaned up. Stops the sediment from growing.
  • Per-run unique race tag as defensive isolation — handles concurrent CI runs and the sliver between cleanup and re-upsert where two runs could see each other's fellowship.

2. custom commands create → get / update / delete — eventual consistency

create_command returns immediately but the read-side (get_command / update_command / delete_command) regularly hits the command "<uuid>" does not exist on a cold shared app because the write hasn't propagated yet.

Fix: wrap each follow-up step in the existing loop_times helper (already used elsewhere in the spec) so each call retries up to 10× with 1s sleeps.

Real fix

Per-PR scratch app via create_app — tracked separately. This PR is a defensive workaround until that lands.

Test plan

  • bundle exec rubocop spec/client_spec.rb — clean
  • CI green on queries users and custom commands across all Ruby matrix versions

@nijeesh-stream nijeesh-stream force-pushed the nijeeshjoshy/chore-fix-queries-users-flake branch 2 times, most recently from 3d0a6ab to bfa34ce Compare May 12, 2026 18:43
@nijeesh-stream nijeesh-stream changed the title chore(spec): isolate queries-users test from shared-app data leak chore(spec): clean up fellowship sediment + isolate races per run May 12, 2026
@nijeesh-stream nijeesh-stream force-pushed the nijeeshjoshy/chore-fix-queries-users-flake branch 2 times, most recently from 430cea6 to 4ab6e20 Compare May 13, 2026 07:02
@nijeesh-stream nijeesh-stream changed the title chore(spec): clean up fellowship sediment + isolate races per run chore(spec): clean fellowship sediment + retry commands eventually May 13, 2026
@nijeesh-stream nijeesh-stream force-pushed the nijeeshjoshy/chore-fix-queries-users-flake branch 4 times, most recently from fd3a24e to a3be0e1 Compare May 13, 2026 10:07
Two flakes in client_spec.rb, both shared-CI app state, neither an
SDK bug:

1. queries-users: before(:all) seeds Frodo / Sam / Gandalf / Legolas
   and hard-deletes them in after(:all). When a CI run dies before
   the after(:all) fires, leftovers stay on the shared app and the
   next run's queries-users test (race=Hobbit, length==2) picks up
   sediment.

   Fix:
   - Extract cleanup into a cleanup_fellowship_leftovers!(client)
     helper, called from a dedicated before(:all) block so the
     concern is separate from the fellowship setup.
   - Keep the per-run unique race tag as defensive isolation for
     concurrent runs racing the cleanup -> upsert window.

2. custom commands create -> get / update / delete: backend write is
   eventually consistent; the read-side regularly hits
   'the command "<uuid>" does not exist'.

   Fix: add an RSpec block matcher

     expect { ... }.to eventually(matches?(value))

   with chainable .with_retries(n) / .every(seconds). Replace the
   custom-commands loop_times calls with this matcher so the read-side
   reads as an eventual-consistency contract instead of a retry loop.

The existing loop_times helper stays as a backward-compat alias
delegating to a plain  block helper, so other tests that
use it keep working untouched.
@nijeesh-stream nijeesh-stream force-pushed the nijeeshjoshy/chore-fix-queries-users-flake branch from a3be0e1 to 4eb607f Compare May 13, 2026 10:27
The default rspec progress (dots) hides which test is in flight when a
runner stalls. Last week's CI on Ruby 3.4 spent 41 minutes silently
between two examples — the log only shows DEPRECATION warnings before
and after, so the culprit was unidentifiable.

Adds .rspec config (--format documentation, --profile 25) and a
support/timing.rb hook that warns:

  [timing] start <full description>
  [timing] N.NN s <pass|fail> <full description>

on each example. Two effects:

* A hung runner now leaves a 'start ...' breadcrumb so the next person
  knows exactly which spec was mid-flight.
* End-of-run --profile output lists the 25 slowest, which surfaces the
  flake culprits without having to scan the whole transcript.
Comment thread .rspec
@nijeesh-stream nijeesh-stream enabled auto-merge (squash) May 13, 2026 11:50
@nijeesh-stream nijeesh-stream merged commit f3c8f5b into master May 13, 2026
5 checks passed
@nijeesh-stream nijeesh-stream deleted the nijeeshjoshy/chore-fix-queries-users-flake branch May 13, 2026 11:50
@github-actions github-actions Bot mentioned this pull request May 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants